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MOVING DISTRICT REFORM INTO SCHOOLS: LINKS 
BETWEEN TEACHERS’ PERCEPTIONS OF DISTRICT- WIDE 
REFORM EFFORTS, PARTICIPATION IN PROFESSIONAL 
ACTIVITIES, AND STUDENT ACHIEVEMENT 

A nationwide call for accountability in education sprang from the 
publication oi A Nation at Risk (National Commission on Excellence, 1983). 
The resulting move to high-stakes, statewide tests has sent educators scram- 
bling for a recipe for reform that will raise student achievement. 

Unfortunately, the most recent National Assessment of Educational 
Progress (NAEP) reading scores in the critical elementary school years have 
remained flat (Donahue, Finnegan, Lutkus, Allen, & Campbell, 2001). Simi- 
larly, in science and mathematics, American schools are also not meeting their 
goals based on a review of test scores on multiple assessments such as NAEP 
and the Third International Mathematics and Science Study (Schmidt, 2002). 

Currently, the pressure to reform schools has grown even stronger. 
George W. Bush’s No Child Left Behind Act (NCLB) requires that all states 
test all kindergarten through eighth-grade students yearly in mathematics and 
reading and that states provide sanctions for schools that do not improve. 
These sanctions include the prospect of mandated school choice and could 
eventually close some failing schools (No Child Left Behind, 2002). 

Schools in urban districts may have more difficulty meeting the re- 
quirements of NCLB than schools in suburban districts have. NCLB mandates 
that states set goals for school improvement based on current averages on 
state tests and then increase these goals over time (No Child Left Behind, 
2002). Urban schools, which typically score below average on state tests, 
have more ground to make up in order to meet state goals. Additionally, these 
schools will be forced to make immediate progress, while higher performing 
schools may be above state goals for many years regardless of whether or not 
they make efforts to improve (Linn, 2003). As a result, urban schools are 
likely to need immediate reform in order to meet state goals. 

Through “effective school” research beginning with Edmonds (1979), 
school administrators have gained a great deal of information about what types 
of reforms are effective in raising achievement and have learned that teachers 
are crucial to implementing change (e.g., Bath et ak, 1999; Borman et al., 
2000; Carter, 2000; Cotton, 1995; Dolejs & Jenkins, 2001; Elmore & Burney, 
1997; Miller-Whitehead, 2001; Trimble, 2002; Wagstaff, Melton, Lawless, & 
Combs, 1998). This body of research has been seen as a direct contradiction 
to earlier research such as Coleman et al. (1966), which postulated that chil- 
dren in poverty would never be able to compete with wealthier peers (Teddlie 
& Reynolds, 2000). However, despite much evidence that schools can be 
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effective for chiidren in impoverished areas, iittie is known about how to 
change ineffective schoois into effective schoois. Although districts use pro- 
fessional development, principal leadership, and teacher leadership to help lev- 
erage reform, these efforts have varied from district to district and school to 
school. 

For instance, over the past four years in Ohio, the Cleveland Munici- 
pal School District (CMSD), a Title 1 district, has launched a series of reform 
initiatives aimed at improving school effectiveness, particularly targeting el- 
ementary schools. However, as in many urban school districts serving chil- 
dren placed at risk, acceptance and ownership of reform efforts by teachers 
cannot be taken for granted. In addition, the extent to which these efforts lead 
to measurable improvement in student outcomes may be questionable. 

Nationwide, accountability is often measured by students’ standard- 
ized test scores. In Ohio, the state report card is based primarily on the per- 
centage of students passing state proficiency tests in the fourth, sixth, and 
ninth grades. In 1997 when testing legislation first went into effect in Ohio, 
fourth graders who did not pass the fourth-grade reading test were slated to 
repeat fourth grade starting in 2001. Although the law was revised before the 
mandatory retention rule went into effect, the legislation resulted in many 
districts placing great emphasis on elementary-school reform, focused par- 
ticularly on the Ohio Fourth-grade Proficiency Test. 

The state report card system also originally classified only districts, 
not individual schools, into four categories: effective, continuous improve- 
ment, academic watch, and academic emergency. Urban districts in particular 
fared poorly on the report card. Now, with the revised state report card sys- 
tem, Ohio will also rate individual school buildings using this method and, 
thus, districts are more likely to hold principals and teachers accountable based 
on student passage rates. In order to meet the requirements of NCLB the state 
will also place special emphasis on reading and math tests in elementary school. 

Reform Efforts in Cleveland Municipal School District 

In 1998, when CMSD was placed under the control of the city gov- 
ernment, the mayor-appointed C.E.O. for this large urban school district faced 
significant challenges. For instance, according to data from the district’s Of- 
fice of Research and Evaluation (1998), the poverty rate was high. Approxi- 
mately 64% of students received free or reduced-price lunch, an indicator of 
poverty, and 73% of students came from single-parent homes. The mobility 
rate was also high. About 16% of students had changed schools at least once 
during the year. These and many other school and community characteristics 
presented a major challenge to the district’s administration, since such vari- 
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ables have been associated with low academic achievement (Bums, Lippman, 
& McArthur, 2001). Indeed in 1998, the district did not pass any of the state’s 
18 standards for academic achievement (Ohio Department of Education, 1999). 

To meet these challenges, the district’s central administration intro- 
duced a collection of district-wide reforms primarily targeting elementary 
schools. These reforms are now included in its vision statement and updated 
in its annual reports. Though not part of the official movement, many of this 
urban district’s reforms are consistent with those of the “effective schools” 
movement, as presented in a recent update (Taylor, 2001). Reform initiatives 
implemented by the district include: Implementing academic standards, in- 
volving parents, engaging in meaningful professional development, providing 
a safe and orderly environment, emphasizing basic skills, involving the com- 
munity, increasing student literacy, working from a clear and focused mis- 
sion, having high expectations for all students, monitoring student progress 
frequently, using data-based decision-making, maximizing student time on task, 
involving teachers in decision-making, and providing instmctional leadership 
for teachers. However, in the CMSD, passage rates for the state proficiency 
test scores have varied radically, from schools that have made little or no 
improvement since 1998 to schools that have increased their passage rates to 
exceed state averages. 

Teachers’ perceptions of reforms also vary, from those who say their 
schools have made no progress in implementing reforms to those who say 
their schools are getting much better. Teacher perception of elements of ef- 
fective schools has been shown to correlate with student academic achieve- 
ment (Blust, 1986; Charlton, 1989; Sabatella, 1992; Wilson, Firestone, & 
Herriott, 1985). Studies have also demonstrated the important role teachers 
play in raising student achievement (Sanders, 1998; Sanders & Horn, 1998). 
In some schools, particularly in urban communities, variation in teacher per- 
ception of school reform and in fluctuation in test scores may suggest that 
centralized reform efforts have been ineffectively disseminated to teachers, 
making implementation of these reforms difficult. It is therefore of paramount 
importance that teachers should not only play a leading role in any school 
reform effort but also take ownership of these efforts. In a study of three 
large, urban districts, Corcoran, Fuhrman, and Belcher (2001) suggested that 
large districts have particular difficulty working district- wide reforms into the 
fabric of daily classroom life. 

Several recent studies point to the use of instmctional leadership — in 
the form of professional development workshops, principal supervision, and/ 
or participation in inter and intra-school school teams — to put reforms into 
practice in high-poverty and/or high-minority schools (Borman et al., 2000; 
Carter, 2000; Dolejs & Jenkins, 2001; Miller-Whitehead, 2001; Trimble, 2002). 
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Such strategies could be critical in large, urban districts (Elmore & Burney, 
1997). However, these studies do not always specifically address teachers’ 
perceptions of reform efforts and do not specifically relate these perceptions 
to student test score gains over time. 

Purpose of the Study 

This study attempts to relate variations in school passage rates on the 
Ohio Fourth-grade Proficiency Test between 1999 and 2002 to the extent to 
which teachers are incorporated into the district reform efforts. Participation 
in professional development and collaboration with educational leaders, such 
as principals and lead teachers, have been two major strategies to incorporate 
teachers into the district reform efforts. The degree to which these practices 
have been effective has varied from school to school. 

The primary purpose of the study is twofold. First, it examines the 
connection between teachers’ participation in professional collaboration ac- 
tivities and in professional development and their perception of the effective- 
ness of the district reform efforts. Second, it aims to determine the extent to 
which teachers’ assessment of the effectiveness of school reform efforts can 
lead to school improvement in the passage rates on the state standardized 
tests. In particular, the study addresses the following specific research ques- 
tions: 

1. To what extent do teachers’ participation in professional develop- 
ment activities and engagement in professional collaboration predict 
the perceived success of their schools’ reform efforts? 

2. To what extent does the success of the school’s reform efforts as 
perceived by teachers predict the school’s improvement on math and 
reading passage rates for the Ohio Fourth-grade Proficiency Test 
between 1999 and 2002? 

Methodology 


Subjects 


A total of 620 teachers from 82 elementary schools in the CMSD 
participated in this study. These respondents represented 20.9 % of the 2960 
who were contacted to participate in the study. Though the sample respon- 
dents do not constitute a random sample of elementary school teachers from 
the district, there is indication that they were a fair representation of the teach- 
ers in the district, particularly in terms of gender and ethnicity (see Table 1). 
All grade levels and subjects were represented. Respondents’ mean years ex- 
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perience was 4.0 on a five point scale in which l=less than 1 year, 2=1-3 
years, 3=4-9 years 4=10-19 years, 5=20 or more years. The mean years of 
experience in the CMSD was 10.2 years, according to state records (Ohio 
Department of Education, 2002). 

Table 1 

Participants in the Study Compared to All Teachers in the District by 
Gender and Race 




Sample participants 

Population in district 

Characteristics of participants 

Number 

Percent 

Number 

Percent 

Gender 

Female 

554 

89.4 

2396 

87.7 


Male 

49 

7.9 

337 

12.3 


Unidentified 

17 

2.7 

n/a 

n/a 

Race 

Asian 

8 

1.3 

15 

.5 


Black 

99 

16.0 

674 

24.7 


Fatino 

10 

1.6 

78 

3.0 


White 

450 

72.6 

1962 

71.8 


Other 

13 

2.1 

n/a 

n/a 


Unidentified 

40 

6.5 

n/a 

n/a 


Instrument 

The survey instrument used in the study sought teacher responses in 
three areas. First respondents were asked to indicate the number of times per 
year they participated in specific professional collahoration activities using the 
scale: (1) zero times per year, (2) one to four times per year, (3) five to nine 
times per year, (4) monthly, (5) weekly, or (6) daily. Professional collahora- 
tion activities included meeting with teachers to plan and share, team teaching, 
being evaluated by a principal, being observed by a principal, attending a teacher- 
led workshop, consulting with a teacher leader, and getting materials from a 
teacher leader. 
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Second, teachers were asked to rate how well their school was imple- 
menting district reforms. The areas of reform used in this case were similar to 
those constituting qualities of effective schools. Responses in this area used a 
four-point scale: (1) getting worse, (2) always had, (3) getting better, and (4) 
always good. 

Finally, teachers were asked to indicate how much professional de- 
velopment they had received on particular reform-related areas using the scale: 
(1) none, (2) not enough, or (3) just enough. Professional development topics 
included: academic standards, data-based decision-making, classroom man- 
agement, parent and community involvement, safety, literacy, and social ser- 
vices for students. The survey also collected teachers’ basic demographic 
characteristics such as gender, ethnicity, years of experience, grade level(s) 
taught, subject matter taught, and name of school. 

All survey items were developed based on the stated policies and 
reported activities of the district and based on the literature on school effec- 
tiveness. In order to further refine the instrument, five teachers in the district 
reviewed the survey and helped revise it for clarity. After data were collected, 
the internal consistency measure of reliability was established using the 
Cronbach Alpha. Professional collaboration, which consisted of six items, had 
an alpha of 0.67. Ratings of district reform efforts, which consisted of 15 
items, had an alpha of 0.93. Professional development, which consisted of 7 
items, had an alpha of 0.82. 

Data Collection Procedure 

A total of 2960 surveys were sent to elementary school teachers’ 
home mailing addresses in March, April, and again in June of 2002. In late 
April, 247 surveys were also hand-delivered to teachers by community or 
teacher volunteers. In mid June through July, 350 phone calls were made to 
teachers encouraging them to return the surveys. These calls targeted schools 
that had low return rates. In mid June, the survey was also posted on a web 
site; however, only two elementary-school teachers responded via e-mail. A 
total of 620 (approximately 20.9%) responded. 

School building passage rates in the Ohio Fourth-grade Proficiency 
Tests for the years 1999 through 2002 were obtained from the Ohio Depart- 
ment of Education (ODE). The state reports the percentage of students pass- 
ing state tests each year. The year 1999 was chosen as an initial point since the 
management of CMSD was transferred from the elected school board to the 
city mayor in the fall of 1998. 
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Data Analysis 

Data from the study were analyzed in two phases. In phase one, 
descriptive statistics in the form of means and standard deviations were used 
to summarize teachers’ perceptions in the three areas: success of district re- 
form efforts in their school, level of participation in professional collaboration, 
and amount of participation in professional development. A multiple linear re- 
gression model was used to determine the extent to which participation in 
professional development activities and engagement in professional collabora- 
tion could predict teachers’ perceptions of the success of district reform ef- 
forts in their building. All three variables in this model were continuous and 
represented teacher-level information. The two independent variables were 
“professional collaboration,” which was aggregated from six items (alpha of 
0.67), and “professional development,” which was aggregated from seven 
items (alpha of 0.82). The dependent variable “ratings of district reform ef- 
forts” was aggregated from 15 items (alpha of 0.93). 

In phase two, an individual growth model (Raudenbush & Biyk, 2002; 
Rogosa, Brandt, & Zimowski, 1982; Rogosa & Willett, 1983, 1985) was used 
to determine the extent to which teachers’ perceptions of school reform could 
predict the rate of improvement in the school passage rate on the Ohio Fourth- 
grade Proficiency Test between 1999 and 2002. Success on the Ohio Fourth- 
grade Proficiency is reported at the school level. Therefore, in order to obtain 
an overall measure of teachers’ perceptions of reforms, individual teachers’ 
ratings of the implementation of reforms were aggregated at the school level 
and used as the independent variable predicting growth in the school passage 
rate on the Ohio Fourth-grade Proficiency. The number of teachers represent- 
ing each school ranged from 2 to 20, with an average of eight teachers per 
school and a standard deviation of 3.9. The resulting continuous variable had 
a mean of 3.0 and a standard deviation of 0.35. Due to changing criteria for 
passing reading and math tests from year to year, passage rates for each of the 
four years were transformed into standard scores. An individual growth model 
(Raudenbush & Bryk, 2002) was used to assess the extent to which the per- 
ceived success of a school’s reform efforts could predict that school’s rate of 
improvement in mathematics and reading passage rates between 1999 and 
2002. 

Model Specifications 

The within-school models for both math and reading provided the 
growth trajectory for each school for passage rates between 1999 and 2002. 
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in which Y.^ is the standard score passage rate for school i at time t, is the 
initial status of school i, and is the rate of improvement in standard devia- 
tions per year. At level-2, the impact of school reform efforts on the initial 
status of schools (Tig.) and on the rate of improvement in school passage rate 
(tTj. ) is assessed using the linear equations, 

\-/^oo + /^oi(REFORM,) + r„,. 

^i,-/^io + Ai(REFORM^) + r„ 

where, ^gg is the predicted initial status for a school whose reform effort was 
rated as zero (y-intercept), gives the strength of the reform efforts on the 
initial status of the school, jS^g is the predicted rate of improvement in passage 
rate per year for a school whose reform effort was rated as zero (y-intercept), 
and jSjj is the impact of reform efforts on the rate of improvement (accelera- 
tion parameter). 


Findings 


Phase One 

In phase one of the analysis, teachers’ individual ratings of reform 
and professional activities were examined. Teachers’ perceptions of reform 
efforts, of engagement in professional collaboration, and of participation in 
professional development activities were summarized using means and stan- 
dard deviations. These summary statistics are presented in Table 2. The mean 
teacher rating for “district reform efforts” was 3.0, indicating that on average 
teachers viewed their schools as getting better. The mean teacher rating for 
“engaging in professional collaboration” was 2.9 indicating that on average 
teachers collaborated about five to nine times per year. The mean teacher 
rating for “professional development” was 2.2, indicating that on average teach- 
ers felt they had not had enough professional development. The standard de- 
viations for these three variables ranged from 0.5 to 0.8, indicating that teach- 
ers’ answers varied only slightly. 

A linear multiple regression was used to examine the extent to which 
teachers’ perceptions of their school’s reform efforts can be predicted by 
participation in professional development activities and engagement in profes- 
sional collaboration. The results revealed that teachers’ perceptions of school 
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reform efforts were positively predicted by their engagement in professional 
collaboration (j8 = 0.329, p < 0.001) and their participation in professional 
development activities (/I = 0.358, p < 0.001). The predictors combined ac- 
counted for approximately 30.8% of variance in their perception of the suc- 
cess of school reform efforts. 

Table 2 

Means and Standard Deviations of Teacher Perceptions of Reform and 
Professional Activities 


Teacher perceptions of . . . 

Mean 

SD 

. . . district reform efforts 

3.00 

.60 

(l=getting worse, 2=always bad, 3=getting better, 

4=always good) 



. . . engagement in professional collaboration 
(l=none, 2=two to four times per year, 

3=five to nine times per year, 4=monthly, 5=weekly, 6=daily) 

2.90 

.79 

. . . participation in professional development activities 
(l=none, 2=not enough, 3=just enough) 

2.16 

.52 


Phase Two 

In phase two based on the individual growth model, the predicted 
overall initial passage rate in 1999 was 36.32% for reading and 35.40% for 
mathematics. On average, schools improved in passage rate by 0.02904 stan- 
dard deviations or 0.49 percentage points per year in reading and by 0.03091 
standard deviations or 0.62 percentage points per year in mathematics. This 
rate of growth falls far short of meeting the demands of NCLB, which re- 
quires schools and districts to achieve 100% passage rates by 2014 (No Child 
Left Behind, 2002). However, teachers’ perceptions of the success of district 
reform efforts in their school building had a statistically significant positive 
impact in predicting the rate of improvement in school passage rates between 
1999 and 2002 in both reading (jd - 0.29, p < 0.01) and mathematics (fd = 
0.36, p < 0.01). On the survey’s four point scale, every one point increase in 
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teachers’ averaged perceptions of the success of reform effort is predicted to 
result in a 4.87 percentage points increase in their school’s passage rate in 
reading and a 7.17 percentage point increase in its passage rate in mathemat- 
ics. The initial status of a school’s passage rate in reading and math was not 
significantly predicted by the teachers’ perceptions of the success of reform 
efforts in schools (see Table 3). 

Table 3 

Individual Growth Model Results for the Prediction of Improvement in 
School Passage Rate on the Ohio Fourth-Grade Proficiency Test Between 
1999 and 2002 by School Reform Efforts 



Indicators of school passage rates 


Initial status (tt^) 

Rate of change (nf) 

School reform efforts 

Coefficient 

P-value 

Coefficient 

P-value 

Math reform 

0.3482 

0.304 

0.3569 

0.000 

Reading reform 

0.4389 

0.207 

0.2891 

0.002 


In order to illustrate graphically the impact of school district reform 
efforts, as perceived by teachers, on the rate of school improvement in the 
passage rate on the Ohio Fourth-grade Proficiency Test, schools were classi- 
fied as reformers and non-reformers. Reformer schools were those whose 
average teacher rating of how well their school was implementing reforms 
was above 3.0. Those schools with an average rating of no more than 3.0 
were classified as non-reformers. Figures 1 and 2 present the predicted linear 
trajectories for reformer and non-reformer schools between 1999 and 2002 
for mathematics and reading respectively. 

As illustrated in the two figures, though both reformer and non-re- 
former schools started near the same point in 1999, reformer schools have 
experienced gradual improvement in passage rates in both mathematics and 
reading while non-reformers have experienced a slight decline between 1999 
and 2002. Initially in mathematics, the gap in passage rate was 5.6 percentage 
points; four years later the gap had increased to 20.2 percentage points. Simi- 
larly in reading, the gap had increased from 4.6 percentage points in 1999 to 
15.8 in 2002. 
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Mathematics 



Fteformers 

l\bn-reformers 


Figure 1: Average predicted linear growth trajectory in mathematics passage 
rates for reformer and non-reformer schools, 1999-2002. 



Figure 2: Average predicted linear growth trajectory in reading passage rates 
for reformer and non-reformer schools, 1999-2002. 
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Conclusion and Discussion 


Findings show that teachers’ perceptions of the success of reform 
significantly predict improvement in student passage rates in both mathemat- 
ics and reading tests. When teachers reported more successful school-wide 
implementation of reform efforts, passage rates accelerated. When teachers 
reported less successful school-wide implementation of reform efforts, pas- 
sage rates declined slightly. Thus, how well reforms are embraced and imple- 
mented by school staff appears to have a significant impact on student aca- 
demic success. Working well-researched reform initiatives into school culture 
is likely to increase test scores based on this analysis of the CMSD. 

The study also provides information on how districts can work re- 
forms into school culture. Findings indicated that adequate professional devel- 
opment specifically focused on reforms and more frequent professional col- 
laboration with principals and other teachers are positively related to increas- 
ing teachers’ ratings of the success of reform efforts in their school buildings. 
However, as in many urban districts, not all schools stmcture their time around 
these professional activities. Indeed on average, teachers in this study re- 
ported that they did not get enough professional development and that they 
collaborated with other staff less than once a month. Increasing targeted pro- 
fessional development and professional collaboration at schools in this dis- 
trict, and perhaps at schools in many urban settings, is promising and could 
increase reform efforts, which will ultimately improve students’ learning. 

Certainly, much could be gained by schools and students in terms of 
academic achievement if professional activities were increased. However, even 
if the increases in passage rates predicted by this study continued in future 
years, a typical “reformer” school in this study would probably not meet the 
goals of NCLB. By 2014, NCLB demands that, in all schools, 100% of all 
general education kindergarten through eighth-graders pass state achievement 
tests in math and reading, as well as 100% of all kindergarten through eighth- 
grade students in all raciaFethnic groups, in special education, and in English 
language learners classes (No Child Left Behind, 2002). By 2014 in a typical 
“reformer” school in this study, 91.2% of students would be predicted to pass 
the math portion of the state test while 75. 11% would be predicted to pass the 
reading portion. 

Still, because this study shows that four years of concerted reform 
efforts in schools with high levels of professional development and collabora- 
tion correspond with increases in passage rates, the findings could reasonably 
be used to argue that reforms can work in urban schools given enough pro- 
fessional development and teacher collaboration. Educators could reasonably 
advocate increasing these activities in schools where reforms have not caught 
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on. However, critics might counter that even these efforts are not likely to 
ensure that all schools meet the stringent goals of NCLB. A recent examination 
of NCLB hy the National Center for Research on Evaluation, Standards, and 
Student Testing concludes that NCLB requirements, particularly for lower 
performing schools in urban areas, may he impossible to meet (Linn, 2003). 
Given this projected difficulty, it is not surprising that even CMSD’s most 
promising schools are not predicted to reach NCLB goals. Therefore, perhaps 
a compromise position is necessary. Because professional development and 
collaboration are likely to increase teachers’ perceptions of the effectiveness 
of school reform efforts and, in turn, increase student passage rates on tests, 
these reform efforts should be continued and expanded. However, new solu- 
tions for increasing student achievement must also continue in the CMSD, as 
well as in other urban schools where similarly low student achievement threatens 
to keep schools from meeting the goals of NCLB and, more importantly, shows 
a failure to provide adequate education for students. 
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